Phase-Stretch Adaptive Gradient-Field Extractor (PAGE)

Emulated by an algorithm, certain physical phenomena have useful properties for image transformation. For example, image denoising can be achieved by propagating the image through the heat diffusion equation. Different stages of the temporal evolution represent a multiscale embedding of the image. Stimulated by the photonic time stretch, a realtime data acquisition technology, the Phase Stretch Transform (PST) emulates 2D propagation through a medium with group velocity dispersion, followed by coherent (phase) detection. The algorithm performs exceptionally well as an edge and texture extractor, in particular in visually impaired images. Here, we introduce a decomposition method that is metaphorically analogous to birefringent diffractive propagation. This decomposition method, which we term as Phase-stretch Adaptive Gradient-field Extractor (PAGE) embeds the original image into a set of feature maps that selects semantic information at different scale, orientation, and spatial frequency. We demonstrate applications of this algorithm in edge detection and extraction of semantic information from medical images, electron microscopy images of semiconductor circuits, optical characters and finger print images. The code for this algorithm is available here (https://github.com/JalaliLabUCLA)

PhyCV: The First Physics-inspired Computer Vision Library

PhyCV is the first computer vision library which utilizes algorithms directly derived from the equations of physics governing physical phenomena. The algorithms appearing in the current release emulate, in a metaphoric sense, the propagation of light through a physical medium with natural and engineered diffractive properties followed by coherent detection. Unlike traditional algorithms that are a sequence of hand-crafted empirical rules or deep learning algorithms that are usually data-driven and computationally heavy, physics-inspired algorithms leverage physical laws of nature as blueprints for inventing algorithms. PhyCV features low-dimensionality and high- efficiency, making it ideal for edge computing applications. We demonstrate real-time video processing on NVIDIA Jetson Nano using PhyCV. In addition, these algorithms have the potential to be implemented in real physical devices for fast and efficient computation in the form of analog computing. The open-sourced code is available at https://github.com/JalaliLabUCLA/phyc

Physics-inspired Computational Imaging for Machine Vision, Drug Development and Cancer Immunotherapy

Traditional algorithms prevalent in computational imaging and signal processingare hand-crafted empirical rules synthesized to achieve a desired goal. In contrast, our approach is to craft qualitatively new algorithms by emulating laws of physics. Here, we show that Non-Linear Schrodinger Equation (NLSE), the master equation in optical physics can be exploited to invent a new class of computational imaging algorithms with best-in-class performance. We demonstrate a new contrast enhancement algorithm that is computationally efficient, achieves superior color gamut performance, and is able to support real-time video enhancement at 4K and 8K resolutions. We also show how the NLSE operator becomes an edge detection algorithm with exceptional performance in low light levels. In certain cases, these algorithms have the potential to be implemented in physical optics. We demonstrate efficacy of these algorithms in solving a variety of problems for different real-world applications. Specially, we have developed CytoLive, an award-winning real-time live cell tracking tool utilizing our NLSE-guided algorithms to analyze time-lapse microscopy videos acquired under low light conditions. This tool preserves inherent cell behavior by overcoming phototoxicity and photobleaching and has the potential for accelerating research in drug discovery. Next, we discuss CytoEye, a cancer immunotherapy toolbox that mitigates the computational overload of analyzing giga-pixel sized pathology images of tumor microenvironment. Quantitative features extracted by this tool have the capability to predict whether or not patients respond to therapy { an important step toward personalized cancer immunotherapy

Decision Support Systems for Radiologists based on Phase Stretch Transform

Phase Stretch Transform (PST) is a physics-inspired computational approach developed in Jalali lab for feature enhancement in images. Here, it is applied to medical images. The results of its application to X-rays leads to development of an assistance tool for diagnosis of pneumothorax in X-ray images. The tool, which is first-of-its-kind, helps in locating the boundary of a collapsed lung, a life-critical clinical examination which is otherwise difficult for a radiologist to locate with a naked eye. Additionally, PST is applied to other medical images, such as histology and mammograms, to demonstrate feature enhancement. The resulting edge detection map offers promising application in segmentation and analysis of medical images which is explored here. Further, texture segmentation using PST is also demonstrated